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ABSTRACT 



This paper outlines a model for multigroup evaluation 
planning in a rural-education setting wherein the commitment to the 
structure necessary to evaluate a program is needed on the part of a 
research and development laboratory, the state departments of 
education, county supervisors, and the rural schools. To bridge the 
gap between basic research, development, and dissemination and 
diffusion in such a setting, a continual 4— cycle evaluation process 
is set forth (such as has been used with the Small Schools Program of 
the Development Division in the Northwest Regional Educational 
Laboratory) - In addition, to the narrative in the document. Chart I 
contains the research and development continuum in terms of 9 phases; 
research, scanning, invention, prototype, adaption, field test, 
redesiqn, diffusion, and adoption; Chart II illustrates the 4 -cycle 
assessment procedure; Chart III shows the basic data flow from the 
field test sites to the development team; and Chart IV gives the time 
line and responsibilities flow. (MJB) 
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This paper will briefly outline the necessary converging on a specific 
area of concern (rural education) and the committment of a group of agencies 
to the structure necessary to evaluate the developmental phases. 

The history of rural education has been one in which efforts to solve 
problems have been administrative, intuitional and the implementing of certain 
practices which appeared to resolve a perceived problem. Many of these 
approaches have been rather unsuccessful except certain obvious administra- 
tive practices# Practically no research has been done on the basic problem 
of the plight of rural communities. 

There have been, three common* approaches to rural problems* The first 
of these is the appointment of a commission or study group to point out the 
problems. An example of this approach is the National Advisory Commission 
on rural poverty (1967) which declared the situation as they found it a 
"national disgrace," The commission found that while our rural population 
is less than 30 per cent of our total, 46 per cent of the nation’s poor — 

19 million ~ live in rural areas. Most of the reductions in our poverty 
population refer to the. urban, .not rural* poor. For American Indians and 
Eskimos, poverty is a universal fact of life. Rural America continues to 
offer grossly inadequate educational-vocational opportunities to its youngsters 
and very low salaries to teachers. And almost unbelievably, some 60 percent u 
of the rural youth who are enrolled in rural vocational and technical schools 
are studying agriculture * — where there are probably the fewest decent job 
opportunities for them in the foreseeable future ■ 
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The National Advisory Commission on Rural Education made recommendations 
such as: 

(1) Every rural elementary school should have access to specialists 
in early childhood education, 

(2) Private foundations and industry should take a more active 
interest in the quality of rural school teachers and set up 
a system of awards for excellence, 

(3) Every child beginning at the age of three should have the 
opportunity to participate in a good preschool ’ program, 

(4) Every needy child should be provided with free books. 

It appears that these kinds of recommendations are relevant but 
rather global in nature. These recommendations illustrate the need for a 
more systematized approach to the identification and solution strategies 
for rural problems. 

The second approach which has been used frequently is various 
administrative reorganizations or practices which do not apply to rural 
areas. An example of this is the tendency to accredit schools on the basis 
of size. You must have a certain number of students to qualify £>r certain 
federal and state programs that are badly needed in rural areas where the 
sparsity of population many times prohibits the school from participating - 

The third approach is intuitional, which can be illustrated by 
transfer of big school ideas to the rural areas- If it is good for the 
large urban: school* by intuition it is applicable to rural schools. 

In- addition to the administrative and intuitional efforts to solve 
problems in rural education* others have taken either a traditional research 
approach or attempted immediate implementation of a promising practical 
scheme:. Unfortunately* neither of these approaches has been very successful 
and have been* tried most often its rtonrural environments, Research projects 
lead to journal articles and reports but rarely change the conduct of 
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education. Practical efforts typically have little research evidence to 
support them , are not easily replicated, and usually result in minimal 
long-term benefit. 

What is needed is a more systematic approach to the bridging of the 
gap betx^een trained researchers, developers and competent practitioners. 

In bridging this gap between research and development as it applies to 
rural areas there appear to be at least three major steps that need to 
be taken. The creation of ideas and theories (basic research) , the appli- 
cation and testing of these creations (development) and the broad scale 
introduction of proven methods or materials (dissemination and diffusion) . 

Also, three basic conditions must be met if this bridging is to 
take place — first, a tangible product must be devt oped and the product 
must be subjected to successive evaluation and revisions, and the testing 
must be done under the conditions in which it will be used. Rarely has 
this strategy been applied to the typical classroom setting involving 
materials, teachers and students. 

The regional laboratories movement which was initiated in the fall 
of 1966 has as its primary mission the institutionalization of development 
as it relates to the solution of problems in education. The Northwest 
Regional Educational Laboratory is a nonprofit corporation which is clearly 
an advocate for improving education, for innovation and change. As a 
result of this committment the Laboratory is in the business of producing 
new alternatives from which educators may choose. These new alternatives 
are produced with the realization that judgments must be made as to the 
scanning, selecting, revision, field testing and other phases of the 
developmental process. 

Quality assurance in all phases of the RD & B process places an 
emphasis on the evaluation process. For example, the Small Schools Program 
of the Development Division in the Northwest Regional Educational Laboratory 
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has as its primary mission the conceptualization and testing of components 
of a total learning environment for youth in isolated rural schools* The 
activities comprising the program are aimed at the use of modern technology 
to provide a wider range of instruction, defining a new role for teachers 
and training for this role, designing an inservice program for administrators, 
introductory systems technology training for staff, planning space that en- 
hances creative learning, and the development and testing of a rural change 
model* 

The theoretical framework for development of strategies which guide 
the activities of the Small Schools Program is provided partly by Rogers 
in his f, Diffus±on of Innovations’ 1 and by the work of Hartenberger through 
his analysis of the regional! ty of the Northwest* Hartenberger and Rogers 
agree that in order for change to take place a situation must be provided. 

The location of the activity must be in the community where you feel the 
changes should be made* Second, there must be well defined goals or ends 
so that the rural people can easily see that there are end products or means 
of attaining the goal. Thirdly, the development team will be regulated by 
the norms of the local area and they must be acutely aware of regional! ty « 

Summarizing, (a) a situation must be provided, 

(b) where goals can be defined, 

(c) being aware of local norms. 

(Dr, Giammatteo , the next speaker, will discuss this issue.) 

The developmental flow of products within the Small Schools Program 
has nine phases and entry can be made at any phase. The phases begin wi :! 
basic research and end with the adoption process. The nine phases are: 

(1) research 

(2) scanning In preparation 

(3) invention 
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(4) prototype 



(5) adaption 



Interim 



(6) field test 



(7) redesign 



(8) diffusion 



Release 



(9) adoption 



(See Chart I for a more complete description of each phase.) 



This paper intends to describe one overriding condition that must be met 
in order for such a process to take place. This condition is the continual 
evaluation that must be built into the process. Quality assurance demands 
that the summative as well as the formative evaluation be planned and 
carried out. 



is formative and summative in nature and the basic purpose of the model is 
to address the evaluation to the developmental and impact phases. In 
evaluating these two aspects the model calls for the following four-cycle 
assessment procedure. (See Chart II) 

Cycle One - examination of the component's administrative 
structure and process 

Cycle Two — : examination of the programmatic processes within 
the component 

Cycle Three - examination of the component's output and impact 
Cycle Four examination of the contextual system within which 
the component operates . 

Within each evaluation cycle there are a number of stages or !, pulses n 
corresponding either to one of the nine phases of the developmental model 
or to a point on a time line. For example, the three pulses of Cycle One 
are tied to developmental phases two, three, four and six while the various 
lycle Four are scheduled by calendar and in accordance with 



The content of the evaluation model for the Small Schools Program 




certain phases 
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CHART I 

RESEARCH AND DEVELOPMENT CONTINUUM FOR PROGRAM 400 
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CHART I (cont.) 

RESEARCH AND DEVELOPMENT CONTINUUM FOR PROGRAM 400 



CHART I 1 

OI’KKATION OK EVALUATION CYCLES I is RLLAT lONSiiJ. P TO THE 
mini: DEVELOPMENTAL PHASES OK A PROC’RAM 400 COMPONENT 



EVALUATION CYC 




DEVELOPMENTAL 

PHASE 



PHASE ONE 

Researcn 
Scanning or 
Selection 



CYCLE ONE 

ADMINISTRATIVE 
STRUCTURE AND 
PROCESS 



CYCLE TWO {CYCLE THREE 



I PROGRAMMATIC : OUTPUTS AND 

; PROCESS , PROGRAM IMPACT 




i First Pulse* 



-Evaluation of 
Feasibility 
-Process 



| PHASE TWO 

Invention or 
[ Retrieval 



i PHASE THREE 



Prototype 
1 Adaptation 



i 



■ PHASE FOUR 

: 

! Field Test 
Redesign 

PHASE FIVE 
Di f fusion 

i ■ 



First Pulse: 

Structure Analysis 
(of initial 
s tructure) 
-Planning 
-Staffing 
-Budge t 

| Second Pulse: 

| Structure Analysis 
! (of cooperative 
j contacts) 

| Process Analysis 
| -Job allocation 
^Supervision, etc* 



! 

i 

Third Pulse: 

Field test sites 

Teachers 

Managers 



Second Pulse: 

Evaluation of 
—Design process 
-Use of experts 
-Test, design 



Third Pulse: 

j Evaluation of 
| -Field testing 
i -Feedback 
■ -Redevelopment 
• -Cooperation and 
j involvement 
j -Workshop process 
I -Selection of 
j participants 

! 

s 

[ 

: Fourth Pulse: 

j Evaluation of. 
-Final system 
-Design of 
materials 



Fifth Pulse: 
Dissemination 
External 
Internal 



! First Pulse: 

Developmental 

model 

| Change model 
| Prototype 
sys terns 
Equipment 
(if any) 



: Second Pulse: 

! 

; Orientation 
| package 
(Systems managers 
( trained) 
Demonstration 
prototype plan 



PHASE SIX 
Adop tiem 




Third Pulse: 

Review of Structure 
and Alternatives 
Review of Procedures 
and Alternatives 



Sixth Pulse: 
Evaluation of 
trial and 
Installation 





Third-Fourth- 
Fifth Pulse: 
Followup: 
Evaluation of 
outputs and 
dies emin a t ion 



CYCLE FOUR 



CONTEXTUAL 

SYSTEM 



First Pulse: 

Institutional : 
Attitude- s 
Se r v i ce needs 
Policies 
Isolation 
Conflict 
Turnove rs 
Priorities 
Change capaci ty 

Second PuJ.se: 

Teacher s : 

Atti tudes 
Skill ltivcJy 
Experien ce 
Training 
Behaviors 
Change capacity 

S tuden ts : 
Attitudes 
Learning skills 
Achievements 
Aspirations 
Behaviors 
Change capaci 1 y 



Third Pulse: 

NWREL Staff : 
Program, staff 
R & E staff 

Participating 
Groups : 

Admin istrators 
Teachers 
Students 
Comm uni ty 

Fourth Pulse: 

Non-participating 

Groups : 

Adminis trators 
Teachers 
School Boards 
Communities 
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One activity within the Small Schools Program is the adaptation 
of the Patterns In Arithmetic materials produced at the F & D Center of 
the University of Wisconsin. The activity enters the developmental 
continuum at phases 4 and 5 (prototype and adaptation.) The basic steps 
taken to implement the activity are as follows i 
X, Development of rationale for program 

A. Research base 

B. Needs assessment 

C. Staffing for implementation 

II. Establish relationship with rural agencies 
A. State Department of Public Instruction 
1* Building interest in program 

2, Selecting coordinator 

3, Selecting site for trial 

4. Contacting county superintendent 

5. Orientation for county and S.D.P.I. personnel 
III. Establishing plans for evaluation 

A. Identify participants at each level 

B. Techniques to derive plan 
1., Forced field analysis 

2. Target group analysis 

3. Establish time line and responsibilities 
G, Rough draft completed 

D. Pilot of all instrumentation and procedures 

E. Feedback to participants for revisions 

F. Field testing and revisions based on data flow 
G* Start dissemination based on model 

At each of these steps in the developmental process the evaluation design 
falls into one of the basic four cycles. 



II 



Chart III shows the basic data flow from the field test sites to 
the development team. Chart IV is the time line and responsibilities flow. 
This briefly outlines the necessary converging on a specific area 
of concern (rural education) and the committment of a group of agencies 
(the Laboratory * the State Departments of Education, county supervisors^ 
and rural schools) and the structure to facilitate evaluation of the 
solution to certain identified problems, A discussion of the effects of 
the cultural setting and the adoption model will be done by Dr. Giammatteo 
who is the next speaker* 

Thank you. 
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